Evolution of cooperation is a robust outcome in the prisoner's dilemma on dynamic 

networks 
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Abstract 

Dynamics of evolutionary games strongly depend on underlying networks. We study the coevolutionary prisoner's dilemma 
in which players change their local networks as well as strategies (i.e., cooperate or defect). This topic has been increasingly 
explored by many researchers. On the basis of active linking dynamics [J. M. Pacheco et al., J. Theor. Biol. 243, 437 
(2006), J. M. Pacheco et al., Phys. Rev. Lett. 97, 258103 (2006)], we show that cooperation is enhanced fairly robustly. In 
particular, cooperation evolves when the payoff of the player is normalized by the number of neighbors; this is not the case in 
the evolutionary prisoner's dilemma on static networks. 

PACS numbers: 



I. INTRODUCTION 

Existing studies have mainly focused on the mech- 
anisms underlying other-regarding behavior in social 
dilemma situations. A prototypical model for studying 
this subject is the prisoner's dilemma game, where each 
player either cooperates or defects. From an egoistic per- 
spective, defection is more lucrative and is the unique 
Nash equilibrium. However, mutual cooperation is more 
profitable for a population. Maintenance and emergence 
of cooperation in the prisoner's dilemma are often ex- 
plained in terms of evolutionary dynamics, where indi- 
viduals imitate successful others in the social dynamics 
nomenclature [H, [Ij- 

Upshots of evolutionary game dynamics generally de- 
pend on the network structure underlying interaction be- 
tween players. In particular, it was recently discovered 
that heterogeneous networks enable cooperation in the 
evolutionary prisoner's dilemma In this scheme, 

a hub (i.e., a player directly connected to many others) 
tends to earn higher payoffs than a player with a small 
degree (i.e., the number of neighbors for a player). Co- 
operation on hubs can be stabilized, whereas defection 
on hubs cannot. Cooperation then propagates from hubs 
to the periphery players. A less-noticed fact underlying 
these results is that the payoff for each player is defined 
as the sum of the payoffs earned by playing against all the 
neighbors. If we divide the payoff by the player's degree 
d, 0, Q : which we call the average payoff, or if we shift 
each element of the payoff matrix by a constant d, [l^l , 
cooperation is not enhanced in heterogeneous networks. 
Therefore, for these definitions, hubs arc not advanta- 
geous in terms of the payoff itself. 

It may be realistic to consider that players dynamically 
change partners. Indeed, when strategics of players and 



the network structure coevolve such that players pref- 
erentially link to coop erators, cooperation on a global 
scale is facilitated [lll - [l6| (also see fl7l - [l9| for reviews). 
Most previous studies on coevolutionary social dilemma 
dynamics on networks focused on the summed payoff 
scheme, in which well-connected players tend to obtain 
a large payoff. In contrast, the present study examines 
the coevolutionary prisoner's dilemma in a more adverse 
condition for cooperators: the average payoff scheme. We 
show that even under the average payoff scheme, coop- 
eration emerges through coevolutionary dynamics, with 
the exception of a specific update rule that is known to 
disfavor cooperation in evolutionary games on static net- 
works. 



II. MODEL 

A. Prisoner's dilemma dynamics 

We consider the prisoner's dilemma game in which each 
player at a node is either a cooperator (C) or a defector 
(D) in each round. We use si 6 {C, D} to denote the 
strategy selected by the ith player. 

We set the payoff matrix to [lal 
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The values represent the payoff that the row player ob- 
tains. We assume the symmetric prisoner's dilemma 
game; the payoff of the column player is determined anal- 
ogously. For example, if player i cooperates and player j 
defects, i and j gain —0.5 and I, respectively. 

In one round, each player plays the prisoner's dilemma 
against all the neighbors. We consider two types of col- 
lective payoff. The first is the summed payoff Pi defined 
for the ith player as the summation of the payoff gained 
over all the neighbors. The second is the average pay- 
off Pi/{ki/{k)), where ki is the degree of node i and 



1 



(k) is the mean degree of the network. The normahza- 
tion factor {k) is used to make the magnitudes of the 
summed and average payoffs similar. On static heteroge- 
neous networks, the summed payoff scheme enhances co- 
operation 3|-l6|, whereas the average payoff scheme does 

not [iSIIir 

We assume that the players alter the strategy accord- 
ing to the so-called Fermi update rule [l6|, 22c 2M i unless 
otherwise stated. According to Fermi rule, each player is 
selected at the rate l/Tg. The selected player, denoted 
as «, chooses one of his/her neighbors, denoted as j. The 
strategy of the jth player replaces that of the ith player 
with probability 1/(1 -I- exp[/3(Pi — Pj)]). Otherwise, the 
strategy of the ith player replaces that of the jth player. 
We set (3 = 1. In the summed payoff scheme, the fraction 
of coopcrators is large for a small /? [l^. The main fo- 
cus of the following numerical simulations is the average 
payoff scheme. 



B. Dynamics of networks 

The network is assumed to evolve in accordance with 
the active linking (AL) model proposed by Pacheco and 
colleagues [13, 111] • We implement the AL model as fol- 
lows: 

1. Each pair of players i and j is selected at the rate 
1/Ta. 

2. If players i and j are not adjacent, we connect them 
with probability a,j • a <,j ■ If nodes i and j are ad- 
jacent, we disconnect them with probability 7si.s ■ 
If fci = 1 or kj = 1, we do not disconnect them to 
keep the network connected. 

We set ac = au = 0.15, 7cc = 0.1, 7cd = 7dc = 0.8, 
and 7dd = 0.32, unless otherwise stated. The values of 
ac and au are smaller than those used in [13, [l^ . As a 
result, relatively sparse networks emerge. This is aimed 
at emphasizing heterogeneity in the degree and exam- 
ining its differential effects in the summed and average 
payoff schemes. 

At the meanficld level, the AL dynamics arc described 

by 



■{a.,a.^(X-7 - X.^s,) - 7s..,^.s,sJ, (2) 



where (si,Sj) = (C, C),(C,D), or (D,D), X^^s^j is the 
number of links that have Si and Sj at the two ends, and 
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N{N -l)/2. 



C. Setup for numerical simulations 

We assume N = 100 players. The players initially 
form the complete graph, unless otherwise stated. The 
strategy update and the AL dynamics are defined as in- 
dependent Poisson processes. Because we focus on the 



stationary state of the coevolutionary dynamics, we set 
Tg = 1 without loss of generality [l^ . We stop each real- 
ization at T°° ~ 200 X max{Ta,Ts} and regard the final 
state as an approximate stationary state. The quantities 
shown in the following sections are the averages over 10 
realizations. 



III. RESULTS 

A. Evolution of cooperation for different payoff 
schemes 

The final fraction of coopcrators is shown in Fig. [Tfa) 
and [IJb) for the summed and average payoff schemes, 
respectively. The results for the summed payoff scheme 
(Fig. [Ija)) arc consistent with those in [l3, [Hi; coop- 
crators flourish if the AL is fast enough relative to the 
strategy update and many of them exist initially. We 
find that cooperation also survives in the average payoff 
scheme (Fig.[IJb)), although it is slightly more difficult to 
maintain than in the case of the summed payoff scheme. 

Because the two payoff schemes coincide when the net- 
work is regular (i.e., homogeneous in the degree), we ex- 
amine the dispersion in the degree. The mean degree of 
the nodes in the final networks is shown in Fig. [TJc) and 
[TJd) for the summed (Fig. [TJa)) and average (Fig. [TJb)) 
payoff schemes, respectively. The coefficient of variance 
(CV) of the degree, defined by Yf^^l {h - {k)f /((fc) N), 
in the final networks is shown for the summed and av- 
erage payoff schemes in Figs.[TJe) andH^f), respectively. 
We obtain CV ~ 1 for both payoff schemes, suggesting 
that the dispersion of the degree is comparable to that 
implied by the Poisson distribution. 

Our main finding in this section is that cooperation is 
maintained regardless of the payoff scheme if coevolution- 
ary dynamics are considered. We examine the robustness 
of this finding in the following. 



B. Heterogeneous networks 

Networks with heterogeneous degree distributions do 
not enhance cooperation if they are static and the av- 
erage payoff scheme is used [3, [3, [1| • Although coevolu- 
tionary dynamics with the average payoff scheme enhance 
cooperation, the Poisson degree distribution revealed in 
Fig. [T] may not be heterogeneous enough to sufficiently 
distinguish between the consequence of the summed pay- 
off scheme and that of the average payoff scheme. There- 
fore, we perform additional numerical simulations with a 
modified AL model that yields more heterogeneous net- 
works. 

We use a variation of the static network model pro- 
posed by Goh and colleagues [13] . In the original network 
model, nodes i and j (1 < i,j < N) are connected with 



probability proportional to WiWj, where Wi 
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obtained network has degree distribution p{k) (x k , 
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FIG. 1: (Color online) Results for the ordinary AL dynamics 
with Fermi update rule in the final state, (a, c, e) Summed 
payoff scheme, (b, d, f) Average payoff scheme, (a), (b) 
Fraction of cooperators. (c), (d) Mean degree, (e), (f) CV of 
the degree distribution. 



where 7 = 1 + l/a. On the basis of this model, we 
modify the AL dynamics by replacing the rate at which 
a link is created between players i and j, i.e. Us^as^ 
by WiWjasiOisJ^. We set the normalization constant 
,yV = 0.0015 so that the average degree in the final net- 
work is comparable to that for the original AL dynamics. 
The rule for removing links remains unchanged. 

The numerical results for this variant of the AL 
model are shown in Fig. [2j The fraction of coopera- 
tion is approximately the same as that of the original 
AL model (Fig. [ija), (b)); cooperation emerges in both 
the summed (Fig. EJ^a)) and average (Fig. EJb)) payoff 
schemes when AL is fast and sufficient cooperators ex- 
ist initially. As planned, the average degree in the final 
networks (Fig. [Ijc), (d)) is comparable to that obtained 
from the original AL model. As shown in Fig. [2l^e, f), the 
degree distribution is considerably more heterogeneous 
than in the case of the original AL model (Fig. [IJe, f)). 
Coevolutionary dynamics in the average payoff scheme 
enhance cooperation for both homogeneous and hetero- 
geneous networks. 



C. Different update rules 

The difference in update rules can drastically affect 
evolutionary dynamics on static networks [25l - l29| . In this 
section, we examine the eff'ects of different update rules 
on coevolutionary dynamics. 

We implement birth-death (BP ) u pdate rule [l^, HO, 
HH, death-birth (DB) update rule |3ll 32|. and a variant 
of the DB rule proposed by Nowak [Sj]. In BD rule, a 
player is selected for reproduction with probability pro- 
portional to the payoff, and his/her strategy is transmit- 
ted to a randomly chosen neighbor. In DB rule, a player, 
selected with equal probability 1/N dies, and the neigh- 
bors compete for reproduction on this node, such that 
the reproduction probability is proportional to the pay- 
off. In Nowak's rule, the reproduction probability of the 
neighbors in the DB rule is assumed to be proportional to 
and {Pi/{ki/ {k))Y for the summed and average pay- 
off schemes, respectively. The case 5 = 1 is equivalent to 
DB rule. We set 5 = 20. 

The final fractions of cooperators for the BD rule are 
shown in Fig.[3][a) and|3l[b) for the summed and average 
payoff schemes, respectively. In this case, it is difficult to 
achive cooperation. This is consistent with the fact that 
cooperation is generally not likely for BD rule in static 
networks, as compared to other update rules [2^|3l|. In 
contrast, in DB rule (Fig. [3] (c, d)) and Nowak's rule 
(Fig. |3](c, f)), cooperation emerges when AL is fast and 
sufficient cooperators exist initially. 



D. Fragile C-C links 

We have set 7cc < Idd m the previous numerical sim- 
ulations on the basis of the intuition that neighboring Cs 
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FIG. 3: (Color online) Final fraction of cooperators for various 
update rules. We use (a, c, e) summed and (b, d, f) average 
payoff schemes. We use BD rule in (a) and (b), DB rule in 
(c) and (d), and Nowak's rule with 5 = 20 in (e) and (f). 



™ 0.1 



0.5 1 0.5 1 

initial %C initial %C 



FIG. 2: (Color online) Results for the modified AL dynamics. 
See the caption of Fig. [l]for the legends. 



may be more willing to remain connected than neigh- 



boring Ds. In this case, Cs tend to have larger degrees 
than Ds. To show that this assumption is not needed 
for enhancing cooperation, we perform numerical simu- 
lations by swapping the values of 7cc and jdd ■ We 
use Fermi update rule and the complete graph as the 
initial network. The results for the two payoff schemes 
are shown in Fig. |4l Although the parameter region for 
enhanced cooperation is relatively small, cooperation is 
viable when AL dynamics are fast enough and sufficient 
Cs exist initially. 
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FIG. 4: (Color online) Final fraction of cooperators when 
ice = 0.32 and idd = 0.1. We use Fermi update rule, (a) 
Summed and (b) average payoff schemes. 
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E. Initially sparse networks 



In the previous sections, we performed simulations 
with the all-to-all connection initially. In this section, we 
examine the case in which the network is sparse in the 
beginning. For Fermi update rule, the final fraction of 
cooperators when the initial network is a random graph 
with mean degree 8 is shown in Fig. [S] The results are 
almost the same as those shown in Fig. [IJa, b). 
IV. CONCLUSIONS 



We have demonstrated that the coevolutionary pris- 
oner's dilemma game promotes cooperation in the av- 
erage payoff scheme by extending the results for the 
summed payoff scheme with AL dynamics [13, [l^ ■ The 
results are robust against the introduction of heteroge- 
neous connectivity inherent in the players, changes in 
the update rule, an increase in the rate of removing C-C 
links, and the density of links in the initial network. 

We remark that Fu and colleagues obtained the re- 
sults similar to ours. Using different link dynamics, they 
showed that cooperation is enhanced under both summed 
and average payoff schemes if the link dynamics are fast 
[3^ . In comparison, we have shown that the comparable 
results also hold true for AL dynamics and that the re- 
sults are robust with respect to the heterogeneity in the 
degree, the update rule, and an increased probability of 
pruning C-C links. 
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